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ABSTFACT 

Using Guttman's facet design analysis., four parallel 
forms of. a multiple-choice test were developed. A' m.gipping sentence, 
logically representing the universe of coatent of a basit^- cardiology 
course, specified the facets of the course hnA the semantic 
structural units linking +hem. The facets- were: cognitive processes, 
disease" priority, specific disease, and clinical judgment-^ stage. 
Developing items from specification strinqs was. simpler and quicker 
than developing items from obiectives yithoUt -stecific guidelines. 
Cue to logistical problems, onlv two of the f ou^sBArallel ^orms 
originally planned were actually used. Although this increased the 
number of times each item was used, it did not violate the 
theoretical methodolog-y , nor the notion that the tests were parallel. 
The tests will be field tested with fourth-year medical students at 
five medical schools, to establish their parallel fdrms reliability*. 
(CP) ^ ^ 
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A Practical Methodology for the Systematic Development 

of Equivalent Multiple-Choice Tests 

• ■ 

Phyllis Blumberg. Ph.Dr \ 
University of .Illinois at the l^edical Center ^» 

♦ ^ • * 

and , 

Joel Felner, fJ^D. 

Emory University Sc^iool of Medicine 

This paper^ discusses how parall/el iru HI pie-choice question tests were 

/ ' ' ' ■ ^ 

developed using Gul^^man's facet design ana»lys1s. Facet design analysis provides 

a logical link between tje Instructional content and the •k^ems because the Items 

are specified by the facet analysis of the. objectives. A mapping sentence was 

'deverf0ped from the facet analysis which defined the universe to be tested. This 

^mapping sentence was used to specify a test blueprint for the pariallel exams and 

also specified the particular dirfienslons of eacK Item. This paper^hows how 

facet design analysis Is a practical way tb develop p^ralleUexa^s. 



A PRACTICAt METHODOLOGY FOR THE SYSTEMATIC DEVELOPMENT • : 

* ♦ * ■ * 

J OF EQUIVALENT MULIIPLE-CHOICE TESTS : 

1 . . . i 

, / OBJECTIVES 

The development of a- detailed blueprint of an exam Inducting the specifications 
of test content and the number, of questions per content area Is an Important step 
in test development^^ When alternate fopns of. an examination are needed, test c^n- 
structors most often use the process of developing a content-by-cognfti ve behavior 
matrix as a way of a priori ijisuring functionally equivalent tests. However* most 
test development reference books (e.g. Popham, 1975 and JhorncLike, 1971) do not 
discuss rules for constructing these matrices or for selecttng items. These ^ / 
matrices may be a product of the test developers concept structure' and their own 
Sdiosyncracies. I Often, sets of items have not been shown to beiequivalent by any 
erapirjcal method (Bormuth, 1970). * 

A number of new approaches have been proposed which attempt to define rules 
for generating items. These item- genera ted mechanisms provide for the construction 
of test^ which ard content valid in a logical as well as judgmental way (Millman, 
% ^74 anjl Martuza, 1977)., One of thesi new approaches, the facet design .approach, 
is employed here to develott.^ equivalent multiple-.choice tests. The 1)urpose of 
this paper is to iilustralSra method of developing alternate forms of tests 
according to logical rules of equivalence of facet design analysis. 



Theoretical Orientation 

I 

Facet design was devefloped by Guttman (1969) for the puri^ose of defining a 
research domain. Facet design sf^clfles the limit of the domain as well as the 
ordering of all of the subcomponents of the domain. It Is a powerful tool In 
defining a p^roblem, structuring relationships within and among variables, and 
dealjng with* the problems of equiva^nce (Jordan, 1978). A system similar to a 
Cartesian coordinate system is used to specify the relationship of subcomponents 
of the domain. Instead of drawing the numerous coordinates on graph paper, a 
mapping sentence is constructed (Engel , 1975). 

^ A mapping sentence provides an integrated system for defining the universe 
of relevant concepts r ItT is composed of two parts: the facets and the phrases 
linking the facets together. ,r-^e facets are categories of relevant content. • 
The phrases 1 inking^esie categories suggest- ttie semantic structural unity which 
integrate all of the facets together. Each facet is defined in terms of specifi 
information whi(;h is presented in a list., A facet element is a particular ex- 
ample from the list of a relevant concept. Figure 1 shows a sample mapping sen- 
tence. Thus, a mapping sentence, through the use of facets artc^ facet elements, 
analyzes t^e domain to be sampled into all o,f its relevant component parts 
CMillman, 3974, Martuza, 1977, and Shye, 1978)'. • 

Insert Figure 1 About Here • ' 

, . ■ * 

One specific element from each facet i^combined to form a facet string. A 
facet string defines a p'articular, segment of the universe. Theoretically, the 
number of possible facet strings is equal to the number of combinations, of all^ 
the coordinates. However, some of th^se facet stri^ngs are nujl sets when real 
data are applied. . ^ 



'"Mmman (1974)„Engel (1975), Marluza (1977)! and Berk (1978) propose the 
use of facet-design analysis for" test-Item construction. Initially the test 
i:onstructor conceptually analyzes the content to be tested by dividing the con- 

« 

tent into Its component parts. The result of this analysis yields a mapping 
sentence of the universe. Specific Items can be generated from facet string 
specifications. 

f • . . • 

, V After reviewing various techniques for test specifications and Item- 

generation, Engel (1975) concluded that the facet-design approach Is the only 

approach which Is explicitly based on a meta- theory of content represents tlve- 

ness.- This advantage of a theoretical base makes this approach fundamentally 

attractive since It 1s operational for constructing multiple-choice tests. 

Facet-design analysis provides a logical link between the Instructional content 

and the multiple-choice Items because the Items are specified by the facet-design 

which follows from the objectives. \Un1ess a theoretically based methodology such 

as facet, design i-s used, there Is no a priort way of justifying- the necessary 

assertion that the Items Included In the parallel forms define the same kiyi* ^ 

behaviors. By constructing test specifications and items on €he basis of a 

facet analysis, it beiomes possible, in principle, to devise parallel, forms in 

terms of content. 



Methodology 

♦ 

Task ^ • ' • 

A test specialist \^as asked to help cardiologists develop four parallel , 
multiple-choice tests examlrllng the content of a ^aslc cardiology course as 
part of a larger research project. This project Involved five medical schools. 
Each test was to be composed of about 100 Items. First the cardiologists 
Identified twenty- three diseases jfhat were pertinent to the students' level of 
learning. Next, these diseases were ranked h|gh, medium or low priority based 
on frequency of occurrence. The test specialist employed facet deslgr) analysis 

« 

1n order to Insure content domain .equivalence. 
^ Th6 test specialist stu(;i1ed the objectives and developed a mapping sentence 
which had four major facets and at least three elements within each facet. 
Figure 2 shows the mapping sentence used for the test speciflcatlo/i and Item 
\ dev^lopmeht. The elements have^been ordered on the following dimensions: 

Facet A - cognitive processes employed which are ordered according to an adapta- 
tion of Gagne's 0970) and Bloom's, et. al. (1^56f ctignltlve hierarchies; ^ 
Fa(iet B - disease categories which are rank^ frpm high to low. priority in terms 
of the objectives; Facet C - specific disease/w^hich are ordered alphabetically; 
and Facet, D - the stages in clinical judgment which are ordered by the chrono- 
logical sequence of clinical judgments. 

^ . * Insert Figure 2 About Here 

Theoretically, the next step would be to list all of the possible 2,592. 
facet s.trings. (There is a total of 39 different coordinates from four facets - 
6 X 3 X 24 X 6.) This was. an unrealistic step especially since many of the 
( facet strings dp not reflect reality. For example, the use of visuaV information 
such as x-rays or electrocardiograms only enter into selected steps such as data 



«4«* 
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gcythering (D^)» interpret laboratory data (D^) and diagnose' patient problems J^) . 
Whi'le computer techniques are availalDle to list the strings, it was decided that* 
1n this particular application, it would be sufficient to allow the c^jntent 
specialists to select the salient strir^gs. 

m * 

Test SpecificaJ:ion < , *^ \ 

One cardiologist determined the overall blueprint for the "exam, for example, 

he- specified ^t about one- third of the items should test the student's ability 

« . . • • ■ 

to describe and interpret chest x-rays, electrocardiograms, phonocardioarajns and 

ecHocardiograms (e.g., A2 and A^). From this general specification of the types 

of desired questions, the particular facet strings were selected which fulfilled 

these general criteria. , 

* • 

Each disease was ranked high, medium or low priority- in the objectives. The 
test specifications stated J:he approximate number of items for each disease for * 
the three ranked priorities. However, the particular disease was not specified 
In the test blueprint. Thus, Facet C, the name of the disease was Incorporated 
'into Facet B In the test specification facet string. 'A. n 

Table 1 shows the test specification blueprint defined In facet strings. 
The three facets used in test specification are: A. cognitive processes required, 
B. priorlzed diseases and D. the step witfvin the clinical process. The number 
after each Facet letter refers to the specific element of the facet as identified 
on Figure 2. Specification A^BgD^ means that items should pertain to,' "Recall of 
factual Information relating to the diagnosing of patients with high priority • 
"diseases, " This specification was used to generate two items. In order to in^re 
that the four tests would be parallel In terms of content, one set of facet 
strings determined the test specification blueprint for all of the examinations. 

The visual information referred to in Facet A^- verbally describe visual 
information and Facet A^- Interpret visual information. includes tl|e u^tf of the 
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following diagnostic tests': ambulatory electrocardioqr^ipWc rfionitoring (Holter 
monitoring) ,^ angiography cardiac catheterization, chest x-ray, cineangiogram- 
* coronary-arterlography , echocardibgram, electrocardiogi^am, impulse tracing, pho- 



nocardlogram, radionuclide imaging Ciycardial-scanning) , and treadmill ^tress . 
test. The same diagnostic test used on one exam was used on the other three 
exams with that facet string. Fqr example, since Exam A required students to 

• * 

interpret a chest x-ray during the data gathering phase from a. patient with 
mitral steno§is. Exams B, -C and D also required students to interpret i:hesC 
x-rays during the data gathering phase from patients with, high priorTty^lseases. 

Facets A, B, and D are used for test specification, whereas Facets A, C, and 
D are used for .item specification. The rationale fpr the use of these facets is 
explained ^rlier. 

^ % 

Insert Table-^l About Here j 
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Item Specification , ^ ' 

It was assumed that at least one Item coiild be written for each disease which 

[ . ^ 

fulfilled thS" criteria of a facjt string in Table 1. For example, the facet 
string for test specification "A-jB^D^" specified that the Item should recall 
factual Information relating to jthe data gathering step on a high' priority disease. 
There were ten high priority diseases. Therefore, at least ten different Items 
could be developed from this faceV .string since It was assumed that knowledge, 
about the ten diseases are Interchangeable. These ten Items could be assigned to 
ten parallel examinations. Since only four parallel examinations were required, 
only fpur^such Items had to be developed. iHowever, for the facet string ."A-jB^D^" 
two Items per exaipl nation were required because of the Item specification developed 
(see Table 1). Thus, eight separate Items had to be developed which required the 

*. 

recall of factual Information pertalhing to.orderlr^ laboratory tests for high 



priority diseases. Preferably these eight Items would each deal with a' separate 

' J- 
high priority disease. 

As indicated in the above paragraph, three facets (A, C, and D) were used for 
the item specification. Since all of the diseases were assigned a priority Crank- 
ing, the priority facet (^) was not necessary when the individual diseases were 
considered in the actual item specification. The test specification facet string 
A^B^D^ resulted in the following ten item specification facet strings since there 
were ten high priority diseases:. 

AjCjD, • A,C,2D, 

A1C5O,. A,C,3D, 
. - A,C7D, AiC,4D, 

A^CgD, , A,C,5D, _ , . 

-At least oae item could be developed from each item specification facfet string. 
Item Development , 

The cardiologists had already begun to develop a pool of items. Only those 
items which met the test specification facet string requirements were selected for 
the pool. In some cases Several items were in aixis tehee. Instead of rejecting 
good items, these parallel items were placed in the pool. 

' After the classification of the existing items it became clear that more items 
were needed to meet the exam specificatibns. The cardiologists were then asked to 
develop items according^to the particular test sjjecifi cation facet strings that 
were lacking the required number of iten\s as shown in^ Table 1. The item pool con- 
tained, sets of items grouped according to .the test specification faucet strings - . 
from which they were derived. 
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Test Development 

Since the caV'dIo legists did. not have the time to. develop four hundred Items, 
they asked the test spec;la11st t6 recflmmend a way of developing useful exams 
using fewer Items. After a discussion with one cardiologist, it^as decided that 
50% of each exam could have ""common items. Next the test specialist assigned 
items to each exam based, upon the test specificati^ table of facet strings 
(Table 1). Items were assigned randomly to one of four exams except with the 
stipulation -that if more than one item was required for that facet string per 
exam, each item in an ^exam from that facet string would be concerned with a 
different (^Isease. A running tally was kept on the litems- aS" they were assl^ed 
to ej^ams in order to insure that there would be the correct number of it^ms for . 
each high, medium, and low priority disease. Frequently items had tp be juggled 
from one exam to, another, y^t^ no matter h(iw the Items were juggled, all of the 
test specification criteria, (e.g.. Table 1 and the frequency of items per disease) 
could not be met with the -existing pool. For example, on one exam there were too 
many items on normal patients and not enough items on aortic stenosis. Therefore, 
/a cardictlogist was asked to develop more items to be placed In specific slots on 
each exam on the basis of item specification facet strings. An *i tern specification 
for each of the four parallel exams was developed, fable 2 shows the item sf^ecifi- 
cation for Exam A. • ^ ' . / 

Insert Table 2 About Here > 

I > •■ , 

i 

Test Use ' , 

The four exams we're pilot tested. During the review proces^ it was decided 
that only two exams would be used due to logistical problems and problems with the 
difficulty or wording of specific items. The itans on these two exams were derived 
from the original four parallel exams. 



j These exams will be used for fourtlj-year medical students in a gardioloay 
rotation at five colleges of medicine In the United States as part of a larger 
experimental, s1{nulated Instructional program. The performarfce of these students * 
on these exams will be analyzed to test the hypothesis that these exams are 
parallel In terms of content- based on the facet design analysis conducted.; The 
Guttman-L1ngo computer /jfnalysls as well as factor analysis will be used to test 
this hypothesis. * 

Discussion • . • . 

Periodically, throughout the test development process, the steps defined. by 
the facet design- analysis were modified. These slight modlflcatl-^ns did not 
saicrlflce the rational approach to test developmient. Ideally, bne should 
initially list all of the possible facet strings and. eliminate the unrealistic 
ones. However, this is' not a pY^actical way of developing test blueprints since 
the number of dimensions yis usually large. The test developers selected facet 
Strings from the mapping sentence without actually M sting all possibilities. 
Since the mapping sentence listed alT of the relevant coordinates, the relevant • 
^ facet strings were easy to visualize. ^ 

^ According to the rigorous interpretation of Guttman's theory, all of, the 
^1 terns should developed once the tftst blueprint defining the facet strings is 
specified (Martiiia, 1977). Yet, realisticallyi this may not. be ^ the case. 
Teachers and test developers often rely on a pool of previously constructed and 
I used 1 terns . ^ 

\^ The test specification facet string used three facets; the- item specification 
facet string used three facets, two of wh^ch were the same as in the test specffi- 
cation facet strings. An alternative approach would have been to develop two 
mapping sentences. The cardiologists believed that the elements in the disease 
category could be considered equivalent provided thart the. other coordinates of the 
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facfet string (e.g.. A, B, and D) weVe the sam#. Pulmonary stenosis and coar'cta- 
ti6n wfre both low priority diseases. Therefore, an item which requires the stu- 
dent to describe t6e findings of an 6lectrocardiogram''from a patient with pulmonary 

Sosis (facet string "A2B2C2QP3) was considered equivalent to a question with the 
requirements except from a patient with coarctation during the Interpretation- 
'of- \4f0rat0ry-data- phase (Facet string k^^^^. This hypothesis will ^be tested 
wheq:«the items are used with students. - 7 

Due to the criterion of the number of questions per disease, which was not in- 
eluded in the test specification facet strings,, the assignment of items to an e?<am 
was not straightforward. If a larger poal of items had been developed, it would 
have been simpler to^^ssign items. Instead, a smaller pool of items was originally 
developed and the cardiologists were asked to develop specific items once the test 
development began. However, the cardiologists agreed that develqpifig items from 
item specification string's was a far simpler and less time consuming task than 
devel(iiiing 'items from objectives without specific guidelines. 

The original intent was to develop four parallel tests. Due to logistical 
problems, and level of difficulty of some items, only two exams were actually used. 
Although this increases the number of time% each used, it does rfot violate 

the theorfetical methodology, nor the notion that the tests are, paral lei . This^ re- 



jj^jsion of the original .plan shows the strength and flexibility of facet analysis 
for tern development in that plans can>^e changed throughout. It would also be 
possible later to construct additional tests using the same item spec;ifi cation as 
the original plan. ^ , , ^ 

Recently, Governor Carey of New York signed a "truth- in- testing" bill w^ich 
permits students to s^e their graded tests and the correct answers. Obviously, 
once students can see the actual exams, these items cannot be used again. Test 
specialists have been able to establish that alternate forms of an exam are 



parallel on the basis of consistent performance of different examinees. If items 
cannot be used repeatedly, then test specialists will need new ways^ to establish 
that arternatlve forms are Indeed, paral lei . The method discussed in this paper is 
a potentially qseful way of m^imizing the probability of obtaining parallel forms 
which can be evaluated later. This facet design analysis might be especially use- 
ful for classroom exams. 

The use of facet design appears to be a practical way to 



systematically de- 



velop equivalent forms of tests based on a logical analysis o-^ the domain to be 
tested. This analysis ensures that tests are representative 6f the learning ob- 
jectives. Since the actual development of ^ mapping sentence is not a time con- 
surging activity, It Is a viable strategy for content domain specification. If 
others agree that the mapping sentence logically represents the universe to be 
tested, then the a priori equivalence of the resulting tests can be establlshecj. 
The development of a large pool of Items Is a straightforward task when a facet 
string defines the content of each. 1tem« Thqs, through the use of facet design, 
there Is a logical justification to the claim that the alternative tests are 
^^qulvalent In terms of content. * • ^ 
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T^sz Seed Sicilian TaaU •:ef-'ne<l 



"ica" Strings 



.Ai3:::2 
Aiaiaa . 

A13105 
AI3IC6 
.413201 
A 13202 
A 13204 . 
■Ai32a5 
vA13206 . " 
A 13301 
A 13302 
'A 13304 
A13305 . 
A133C6 

A23ID1 . ^ 
A23103 
A23201 
A233Q3 
A3301 
A233Dj^ 

Auai02- 
A33ia3 

A3aia4 

A3B20X ^ 
A3322a 
A23224 
A3a20S \ : 
A38206 
A3a3Q2 
A38303 
A3a3C4 ^ 
A38305 



.Numder of Quastlons 
Z 

N 2 
2 



I 

3* • 
2 

2 

I* 

3 

2 

4* 

1 • 
4* 
I 

3* 

2 . 

I. ■ 

I 

I 

2 

I 

2 

I 

I- 

I 

2 

2 
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ricat :iCrfng 

A4ai01 •• 
A48103 . 
A4ai04 

A4a2Dl 
A48203 
A482D4 
.A48205 
A48303 
A48304 

A58101 
A5B102 
ASBIOS 
A5aiD4 
^ A5a2C4 
A53205 
ASa302 
A5B303 
A533Q4 
AS3305 
A53356 

A6ai06 
■jv A6a302 
ff>;ip3304 



T 

2 
1 
2 
I 
1 

1 

1 



1 

2 

2 

2 

2 

1 
r 

« 

2 

2 
2 

2 
1 
1 
1 



■ Tfte thr«s facats ar«: Av <gfp« of -tnul t1ple-cholca quftst^on, 3. pr-forijtlisd jabjactive 
an4 0. thf stap within tht^cjtnic^il process (nodal ./The numoer after each ^acat. Uctar 
rtfars to t:he specific tlament of the facat as identified on figure Z.^^ ^ ' 

*?*or«.than t*wQ questions were desired for these facat strings because, the cardfolo- 
; ^tits wanted to emphasize Interpreting laboratory data, diagnosing diseases and 
pharmaceutical .lanagement regimens generally prescribed. 3y the nature of thesa 
* .questioni ttey had to be lew level typeis of questions, i.e., either recall information 
o verbally describe visual information. 
ERIC * V ■ 
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. JO . 
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.AJCI002 




Alt /D5 




7 7 
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.•^cu i U J 


Jo . 


A 1 r m A 




Alt *.0D2 






AJLZ4u5 


72 . 


A1C24U2 






A ir 1 no 


70 

/J v 


•A TP O /I n o * 

AJC24U2 


AAf'init ^ 


/I T 


A yl P 1 on A 

A4C loU4 


/ 4 . 


AICI0D4 


McU JU J 




A yl P A n A 


\ 75. 


A3C16D5 




A T 


A2L22L)J 


/o . 


A OP cn T 






A c/*rtnn 
A5C9u2 


77 ; 


AaC5u3 




AC 


A '5P 1 no 
A2u iUo 


TQ 

■ /o. 


A A r* on o 
A4C3U3 


Air iHit 




A OP An A . 
A^JL*hJ4 


7Q 


A AP 1 on o 
A4CloD3 




A 7 


ACPI Crt A 

A5Ui504 


SO . 


A CP 1 cn A 

A5C19Q4 


Air 7ni* 


AO 


A CP 1 cn A 

A5Ci504 


81. 


A 1 ^ i An i - ^ ■ 
A1C14U1 




Art 


A CP 1 on A 

A5C 1204 


32. 


AlClQDo 


AIl1<h)J ^ 1 


50 . 


A OP 1 on A 


83. 


A5C22U6 


AaCLauS i 


51 . 


A OP 1 Tn A 

AJCi/04 


A 

84. 


A c/^cnc 

A5CoD5 


AoCl-BuZ^ 




A OP AHC 


C3C 

35 . 


A TPTn A t 

A3C7u4'^ 


Aic;2*Mll 


bo. 


• A "IP 1 ono 

. AoCi2u2 


oc 


A 1 0 /I n c 

A1C2405 


AoCouo^ 


CA 


A CP 1 one 
A5Ci2D5 


37. 


A A p 1 1 n 0 
A4C11D3 


AJCju3 


55. 


A OP 1 on 1 


88. 


A'OP t T n 0 

A2ullu3 


AfiCou4^ 


bo . 


A OP 1 one 

AJC120Q 


89. 


A IP A n 0 
A2u4u3 




5/ . 


A AP 1 (in 1 


on 


AOPOAnC 


A IC 1-005' > 


58. 


A cpoon o 

A5C23DJ 


n 1 
91 . 


A 1 p one 
A1L3D6 


A\C40o 


59. 


A o*^ onn o 

A2C20DJ 


no 

. 92. 


A 1 p7ne 
AlC7u6 


A6C1005 ^ 


60. 


Ao^^cno 

A2CoD3 


n o 

93. 


A 1 poono ' A 
A1C22D2 


A1C5D^ 




A OP 1 nn o 
A2C10D3 


n A 
94. 


A cpoon 1 
A5L2JU1 , 


AoC505 


52. 


A OP 1 nn o 
A2C10D3 


nc 
95 . 


A 1 p on-T 


'AlCSDo 


53. 


' A OP A no 

A2C403 • 


ne 
96 . 


A 1 po one 


A5C16D4 


C A 

54. 


A oP 1 no 
A2U103 


(37 


A OP 1 AnA 




55. 


j\ CP oonc 
A5C22DO 


(30 


AiP'^Ane 


A4C\a05 


66. 


A1C1902 


99. 


A3C1302 


1 




100. 


A3C2303 



3. 
1. 

5 . 
5. 

^ 

3. 
9. 

10. 

u. 

12. 
13. 
14. 

/IS- 
IS. 

la. 

19. 

20. 
21. 
22. 

23. 
24. 

'IS. 

26. 

27. 

28. 
'29. 

30. 

31. 

32V 



■ The thrae facets are: A. type of multiple choice question, C. name 
of disease and D. the step within the alirtica! orocess nfiodel . The number 
after each- facet letter refers to the specicic alemeRt of the" facet as 
identified on Figure 2. • 



TABLK 1 



TEST S^FXIFl CATION TABLK DICFINeS BY FACET STRINCS 



FACET STRING 



NUMBER OF QUESTIONS 



FACEf STRING 



NUMBER OF QUESTIONS 





1 

X 












\ 














A131C6 








i 




IT a'? 0*7 / , 


t 






2 




1 








Asaioi 




2 


A5B LD2 




1 


ASa LQS 










' i. y • 


nwUww 




3 \ 


A53205 


A5a302 




2 


A33303 




4* 


A5a3D4 




T 


ASa3DS 




4* 


^ A5a3C6 




1 


✓ 


.A23303 


3* 

• 


1 ' AfiaiDS 






' A6B LCe 






A6a302 






A633D4 


A3ai04 


1 




A3a2ni 


1 . 


i. 


A33232 


/ • 2 


A:32!2'1. 


1 




A3a2!:s • 


2 




A38226 






A^aaoa 

• A3a3C3 


1 




1 




' A3a3C4 


*2 ■ . 




Aaa305 


2 

• 





, .1 



i 
2 
1 
1 

L 

1 



2 

■ a 

2" 

2 

~2 
2 

2 

. I 
I 
i 



,The thr^ facets are: A. type of multiple-choice question. B. prioritized objectives and 



ree 

4^1 



The number after each facet letter refers 

\ 



C. the^tej) within t^e clinical process model, 
to the specific element of the facet as identified on Figure 2. 
More than two queetiona were desired for these facet strings because the cardiologists wanted 
to emphasise ihterpteting laboratory data, diagnosing disuses and pharmaceutical management 
t -^f-xe^iMsm generally prescribed. By thfe nature of these questions they had to be low level 
l^^f^ ot n^^itiopB*: , either recall information or verbally ^leacribed visual liiform**^^®'* 



TABLE 2 



Exam A Item ySpecif icatlon Table Defined by Facet Strings 



i 



3 ♦ 

/ • 

3. 
9. 
10. 

n. 

12. 
13. 

14 : 

15. 
16. 
17. 
'13. 
1-9. 
20. 
21. 
22. 
23. 
24. 
25. 
26, 
27. 
28. 
29. 
30, 
31, 
32, 
33. 



A4C:5DI 

A2C701 

A2C15D; 

A4C9<104 

A2C1103 

A2C13Q3 

A4C2C04 

A^3D4 

A2C3D3 

A5C15D4 

A1C1D4 

A1C204 

A1C3D4 

A5C2402 

A1C7D6 

AlC2De 

A1CU03 

A5C15D5 

A6Cia02 

A1C24D1 

A6C5D6 

A3C3D3 

A6C5D4 

A1C130Q 

AlClOOS 

A1C406 

A6dOd5 

A1C5D4 

A6C505 

A1CS06 . 

A5C16D4 

A1C2405 

A4Cia05 



34, 


AIC240I 




67. 


A5C11D5 


35.VA3C:0C2 




63. 


A5C23D5 


36. 


A3C1002 




69. 


A IC 705 


37. 


AiC6D4 




"0. 


A5C11D4 


38. 


AlC-104 




71. 


■A1C10D2 


39. 


A3C2405 




■ 72. 


A1C2402 


40. 


A3C102 - 




73. 


A3C2402 


41. 


A4CiaD4 




74. 


A1C1004 


•42.' 


A4C4D4 




75. 


A3CI6D5 


43. 


A2C22C13 




76. 


A2C5D1 


44. 


A5C902 




77. 


A4CS03 
A4a03 


45. 


A2C1D3 


• 


• 78. 


46. 


A3C404 




79-. 


A4Cia03 


47. 


A5ei504 




80, 


A5C1504 


48. 


A5C15D4, 




8L.^A1C14DU 


49. 


A5C12D4 




32, 


A1C10D6 


50. 


AJC1304 


} 


83. 


A5C22D6 


51. 


A3a704 


84: 


A5C5DS 


52. 


A3C405 




35, 


A3C7Q4 ' 


53: 


A^C12D2 




36. 


A1C2405 


\54. 


A5C1205 






A4C1103 


5H'*« 


A3C18D1 




li: 


A2C11D3 


56, 


A3C12D6 




89. 


A2C4D3 


57. 


A4C13D1 




■ 90. 


A3C2405 


58. 


A5Ct303 






A1(;3D6 


^ 59: 


A2C20D3 




^ 92. 


A1C706 


60. 


A2C603 




93. 


A1C22D2 


61, 


A2C10D3 




94. 


A5C23D1 


62, 


A2C10D3 




95, 


A1C301 


63: 


A2C403 




96. 


A1C2306 


64. 


i^2ClD3 




9.7. 


A3C1404 


65, 


A5C2206 




98, 


A1C2406 


66, 


Arc 1902 




99. 


A3Cia02 






100, 


A3C2303 



The three facets arer 
r^dlaease a^d D. the step 
each facet letter refers 
Figure 2, . ^ 



A. type of multiple choice question^ C. name of 
within the clinical process model. v The number after 
to the specific element of the facet as identified on 



FIGURE I 

A SAMPLE MA'PRING SENTENCE (Millman. 1974) 



\ 



•The subject contributes to the 



A. REFERENT 

1. self 

2. mus^c education 

3. general public 

4. students 



the 



B. REFERENTS COMMITMENT 



1. belief 

2. feeling 



C. LEVEL 



that the 



1. eleinentary ^^^^^^ ^^3^^. curriculm 

2. secondary 




D. ACTORS (e.e. musii 
curriculum's) BEHAVIOUR 

1. should be 

2. is 



taught by 



E.- TEACHER 

2.* specialist -purpose. of fulfilling 



F. GENERAL STUDENT* NEEDS 

1. relaxation 

2. means of expression . 

3. breaks from academics ■ 

4. emotional stimulation 

5. self-discipline 

6. fun time 

7. contact with a human 

8. group activity ^- . 

9. uncover unknown talents 
10. public performance* ^ 
M. creative outlet ; 

12. success 

13. bring out shy students 



7 ' 

1. A through f are facets. . * * 

2. Al trfrougj^ A4 are facet elements. 

3. A, Bl. 01, 02, E2, F3 is a facet string. 

4» An. item based on the facet string in #3 above^is: 

Do you believe that the elementary school music 
curriculum is taught by specialists for the pur- 
•Pi^se of giving students a. break from academics? • 



In answering nuUlple 
chokft Items on a cardiology 
exaai* students will 





• 










c. 


Specific Disease 
icute HR 


^ 9 Functional H 




AR 


r 

to Hypertension 




AS 


HISS 




ASO 


^12 HVP 




Angina -'CAO 


*^13 MR 




jCoarctdtlon 


^14 MS 


^7 

C 

8 


Congestive Car- 
diomyopathy 

< Endocarditis 


^15 NoriMl patient 
^16 PDA 



Figure 2 

A, Employ cognitive Process 
^ I recall factual Information 

* 2 verba lly^ describe visual Informajilon 

* 3 Interpret factual l^J^matli^^ 
^ 4 Interpret visual Information 



souriC;es 

apply Infc 
sources « 



^17 Pericarditis 

^Id Pul Hypertcn- 
slon-prlmaq; 

^19 Pulm. EiHboll 

^^20 Pulrfbnary Stenosis 

^21 Rheumatic fever 

^22 Vent Aneurysf% ^ 

*^23 VSD 



wf^n confronted 
with a 



B. Disease priority category 
B 1 high priority 
^ 2 medium priority 
* 3 iow priorty 

7 



v. 



which , 
presents as 



at the 
stage 

Nhen« 



D. The step Involved within the clinical' process 
^1 data gathering 
^2 order laboratory date 
^3 Interpret laboratory data 
^^4 diagnose patients problem 



5 determine management plan 
^6 monitor on- going management 

. ■{ ■ 



Is required 



I 



Test specification Involves Facets A, and 0. 
Item specification Involves Facets A. B. and O: 



